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Summary 

Although modern flight decks now feature so- 
phisticated computer-generated electronic displays, 
the display formats themselves are largely electronic 
renditions of earlier electromechanical instruments. 
New computer graphics capabilities make possible 
large-screen, integrated pictorial formats to improve 
situation awareness, pilot/ vehicle interaction, and 
aircraft safety with the potential for significant oper- 
ational benefits. The purpose of this research was to 
compare the spatial awareness of commercial airline 
pilots on simulated landing approaches using conven- 
tional flight displays with their awareness using ad- 
vanced pictorial “pathway in the sky” displays. An 
extensive simulation study was conducted in which 
16 commercial airline pilots repeatedly performed 
simulated complex microwave landing system (MLS) 
approaches to closely spaced parallel runways with an 
extremely short final segment. Four separate display 
configurations were utilized in the simulated flights: 
a conventional primary flight and navigation display 
with raw guidance data and the Traffic Collision and 
Avoidance System (TCAS) II; the same conventional 
instruments with an active flight director; a 40° field- 
of-view (FOV), integrated, pictorial pathway format 
with TCAS II symbology; and a large-screen 70° 
FOV version of the pictorial display. Scenarios in- 
volving conflicting traffic situation assessments and 
recoveries from flight path offset conditions were used 
to assess spatial awareness (own ship position rela- 
tive to the desired flight route, the runway, and other 
traffic) with the various display formats. The study 
showed that the integrated pictorial displays consis- 
tently provided substantially increased spatial aware- 
ness over the conventional electronic flight informa- 
tion systems (EFIS) display formats. The wider FOV 
pictorial display gave equivalent objective results as 
the narrower pictorial format and subjectively was 
preferred by 14 of the 16 pilots. The other two pilots 
had no preference between the two pictorial formats. 

Introduction 

Advances in future airplane cockpits are being 
made possible by the rapid progress in display me- 
dia, graphics and pictorial displays, computer tech- 
nologies, and human factor methodologies. These 
technologies may enable the design of cockpits with 
improved crew situation awareness and workload, 
safety, and operational efficiency during critical mis- 
sion phases. (See ref. 1.) Government and industry 
research programs have been established to develop 
and apply these technologies. One such program in- 
volves the use of “synthetic vision” to enable subsonic 
transport operations when visibility is restricted and 


to provide the cornerstone technology for more ad- 
vanced airplanes, such as a high-speed civil transport 
that may have limited forward visibility because of 
complex aerodynamic and economic requirements. 

Various studies have been undertaken to assess 
the requirements (ref. 2) and to determine the perfor- 
mance (ref. 3) of synthetic vision systems. One study 
(ref. 4) has indicated numerous potential benefits for 
a future high-speed civil transport in which synthetic 
vision is used instead of lowering the nose during 
landing, taxiing, and takeoff maneuvers. These po- 
tential benefits include improved aerodynamic effi- 
ciency, reduced weight, and as much as a 15-percent 
reduction in takeoff gross weight through reduced fuel 
reserves. Synthetic vision capabilities are defined 
herein as the resourceful merging of imaging sen- 
sors (such as fog-cutting sensors), pictorial graphics 
displays incorporating geographic and feature data- 
bases, and advanced navigational aids (such as the 
differential Global Positioning System). An ever- 
increasing interrelationship between onboard capa- 
bilities and airspace management systems is also 
generally accepted; therefore, higher levels of crew 
situation awareness are required to improve perfor- 
mance and safety. (See ref. 5.) Initial investigations 
are being conducted on cockpit flight displays to op- 
timize the spatial awareness component of situation 
awareness. (See refs. 6 -8.) This paper focuses on 
large-screen, integrated pictorial displays as an ap- 
proach to synthetic vision technology and on opti- 
mizing crew spatial awareness. 

To understand situation awareness (SA) in civil 
transport operations, a definition is necessary. Regal, 
Rogers, arid Boucek (ref. 8) state that SA implies 
“that the pilot has an integrated understanding of 
the factors that will contribute to the safe flying of 
the aircraft under normal or non-normal conditions.” 
As SA increases, “the pilot is increasingly able to 
Think ahead of the aircraft,’ and ... do this for a 
wider variety of situations.” This anticipation entails 
“a knowledge of present states, future goals, and 
the procedures used to get from one to the other.” 
Regal, Rogers, and Boucek go on to expound that, 
for the commercial pilot, another dimension of SA 
involves the individual components. One of the more 
important of these components is spatial awareness, 
which in this paper involves knowledge of the own 
ship position relative to the desired flight route, the 
runway, and the other traffic. 

The objective of the investigation reported herein 
was to evaluate and compare the spatial awareness 
component of pilots using displays representative 
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Figure 1. Spatial awareness study display formats. 


of conventional electronic flight information systems 
(EFIS) with two wide-field-of-view pictorial display 
concepts. (See fig. 1.) Two formats based on a 
Boeing 757 instrumentation layout were used as the 
representative conventional EFIS formats. Four al- 
ternate display concepts were compared. The EFIS 
formats, used as baselines, were identical except that 
one incorporated a flight director (with pitch and roll 
commands displayed on two perpendicular needles in 
the attitude display) and the other forced the pilot to 
employ raw deviation error (instrument landing sys- 
tem localizer and glideslope indicators) without the 
benefit of flight director guidance. Both formats were 
included for calibration purposes, as spatial aware- 
ness was hypothesized to be quite different for the 
two conditions. For example, if the pilot concen- 
trated only on centering the flight director needles, 
awareness of surrounding events might suffer; if the 


pilot employed raw position errors, spacial awareness 
might increase. 

The two pictorial concepts were identical “path- 
way in the sky” formats, varying only in horizon- 
tal fielc-of-view (FOV) presentations of 40° and 70°. 
Pictorial perspective displays with pathway formats 
have been investigated extensively by flight display 
researchers (refs. 6, 7, and 9-15) because of the po- 
tential benefit of enhanced SA. However, in those 
studies, the researchers have not attempted to mea- 
sure th< ' benefits directly. The investigation reported 
in this paper was intended to test the hypothesis of 
gains in spatial awareness from the pictorial aspects 
of the display formats. The investigation was cast 
in terms of a single pilot who employs head-down 
displays. Further explanation of the display formats 
follows in the section “Display Conditions.” 
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Abbreviations 

AGL 

above ground level 

CRT 

cathode ray tube 

DERP 

design-eye reference point 

EFIS 

electronic flight information system 

FMS 

flight management system 

FOV 

field of view 

HUD 

head- up display 

ILS 

instrument landing system 

MLS 

microwave landing system 

ND 

navigation display 

OTW 

out the window 

PFD 

primary flight display 

rrns 

root mean square 

SAL 

standard approach to landing 

TCAS 

Traffic Collision and Avoidance 
System 

VISTAS 

visual imaging simulator for transport 
aircraft systems 

VSI 

vertical speed indicator 


Simulator Description 

The Cockpit Technology Branch at Langley Re- 
search Center has developed a flexible, large-screen 
flight display research system, the VISTAS (visual 
imaging simulator for transport aircraft systems), 
which was utilized in this experiment. The simu- 
lator contains the following elements: the simulator 
visual system (visual system hardware and graph- 
ics generation hardware and software), the aircraft 
mathematical model, and the simulator cockpit. 

Simulator Visual System 

The flexible core of the visual system is embodied 
in dual, full-color, high-resolution cathode ray tube 
(CRT) projectors that are configured to vary the pro- 
jected display aspect ratio by matching the edges and 
overlapping the images from each projector. Each 
projected image is 15 in. high by 20 in. wide (stan- 
dard 3:4 aspect ratio), so a maximum 15- by 40-in. 
image can be achieved. This maximum configura- 
tion was used to present the four display concepts 
for this investigation. The images are generated by 
the dual graphics display generators that operated in 
synchronization and used the same visual database 
to produce a single, large-screen, integrated picture 


(combined by the projection system onto the rear- 
projection screen that serves as the main instrument 
panel for the simulated aircraft). Each generator pro- 
vides image resolutions up to 1280 x 1024 pixels in 
a 60-Hz progressive scan format (per projector). As 
the design-eye reference point (DERP) for transport 
cockpit applications is typically about 28 in., the full 
40-in-wide display provides a maximum 70° FOV. 

Aircraft Mathematical Model 

A simplified six-degree-of- freedom mathematical 
model of a two-engine, medium- weight transport air- 
plane was used in this study. The linear transfer 
functions and gains were obtained empirically to rep- 
resent a fixed-wing generic transport airplane. The 
control system represented a system with a basic-rate 
command without attitude hold. Turbulence was in- 
troduced into the mathematical model through the 
addition of a disturbance component (a summation 
of eight independent sine waves) to the roll rate vari- 
able. The level of turbulence was considered moder- 
ate by the participating pilots. 

Simulator Cockpit 

The visual and interactive control elements of this 
flight display research tool have been integrated into 
a pilot workstation. (See fig. 2.) The pilot work- 
station was configured as the pilot side of a generic 
transport, fixed-wing airplane in which the scat could 
be positioned to place the pilot’s eyes at DERP. The 
workstation also accommodated the dual-head pro- 
jection system and the rear-projection screen that 
simulated the instrument panel. A two-degree-of- 
freedom sidearm hand controller with spring center- 
ing provided pitch and roll inputs to the airplane 
mathematical model. A throttle level provided the 
throttle inputs; typical self-centering rudder pedals 
provided yaw inputs. The display screen (instrument 
panel) was titled to provide a 17° line of sight (from 
horizontal) over the top of the screen, which is typical 
of over-the-glareshield views in most airplanes. The 
screen display surface was set perpendicular to the 
pilot’s light of sight. This workstation was then used 
to explore the advantages and limitations of large- 
screen pictorial, reconfigurable display concepts and 
associated interactive techniques. 

Display Conditions 

This experiment was designed to assess the spatial 
awareness component based on integrated pictorial 
displays compared with conventional EFIS formats. 
The two EFIS displays, utilized as baseline measures, 
differed only in that one lacked the flight director 
command bars. A basic T instrument arrangement 
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Figure 2. VISTAS architecture. 


was used with a rendition of a Boeing 757 primary 
flight display (PFD) over a navigation display (ND). 
(See fig. 3.) The pilot adjusted the ND radius 
scale through a switch on the hand controller that 
incremented continually through a discrete loop of 
the available scales (4 to 50 n.mi. in radius). To 
the left of the PFD was a typical airspeed indicator 
dial and to the right were typical altitude, vertical 
speed, and turn coordinator instruments arranged 
vertically in that order. An innovative and un- 
conventional power indicator was used in all four 
display concepts that integrated engine and ambient 
condition information and displayed actual power 
(including engine spool up) in percent of thrust. 
(See ref. 16.) The power indicator also displayed 
power commanded by the throttle setting and power 
desired by the flight management system (FMS) for 
the programmed approach. 

For the integrated pictorial display formats, a 
computer-generated out-the-window (OTW) scene 
with overlaid head-up display (HUD) symbology was 
presented. (See fig. 4.) One pictorial concept was 
rendered for a 70° FOV and the other for a 40° 
FOV. (See fig. 1.) The OTW portion of the dis- 


play consisted of a pathway-based approach, depicted 
by green goalposts the widths and heights of which 
corresponded to fractions of lateral and vertical in- 
strument landing system (ILS) beam errors (±1/2 
and ±1 dot, respectively, with maximum limits of 
±300 ft in width and ±175 ft in height applied at 
the longer ranges). Also, a tiled roadway consisting 
of 20 tiles was presented within the goalposts to aid in 
vertical path control and to present a speed cue (the 
tiles were 150 ft wide, 30 ft deep, and were spaced 
140 ft apart). When the airplane center of gravity 
passed the closest tile, a new tile replaced it at the 
end of the path such that 20 tiles were always present. 
The HUD symbology provided roll and pitch scales 
(in degrees), vertical airspeed and altitude tapes, and 
a horizontal heading tape. All of the tapes incor- 
porated flight management system (FMS) command 
“bugs.” The heading tape also showed ground track 
and the airspeed tape also showed ground speed. A 
vertical speed indicator was integrated onto the al- 
titude tape as a growing or shrinking barber pole 
with a digital vertical speed tag. (The tag position 
on the altitude scale would denote the altitude at- 
tained in 1 min based on current vertical speed.) 
The central HUD symbology consisted of a diamond 
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Figure 3. Over-and-under arrangement of conventional primary flight and navigation displays with supporting instrumentation. 



Figure 4. Seventy-degree FOV, large-screen, integrated, pictorial display concept. 


that depicted pitch attitude and winged-V symbols 
for instantaneous and predicted flight path vectors. 
The display was attitude centered for an attitude 
rate command control system, although the pilots 
attempted to control the flight path vector. A sec- 
ondary smoked-glass (see-through) ND was on the 
left side of the pictorial displays and basically du- 
plicated the EFIS ND. (Map scale control was pro- 
vided in the same manner as that of the EFIS display 
conditions.) Thus, horizontal situation display infor- 
mation was provided that also depicted traffic within 


the OTW-display FOV (delineated by the acute lines 
about the own ship centerline) as well as traffic out- 
side the FOV. 

To evaluate spatial awareness, scenarios (dis- 
cussed in the section “Situation Awareness Assess- 
ment Tools and Techniques”) were constructed that 
required the use of the Traffic Collision and Avoid- 
ance System (TCAS) II. Therefore, both display 
types (all four conditions) incorporated TCAS sym- 
bology, but the implementation differed with respect 





Conventional with flight director 70° field-of-view pictorial 



(a) TCAS II advisory (yellow circle symbology). 


Conventional with flight director 70° field-of-view pictorial 



(b) TCAS II resolution (red square symbology). 
Figure 5. TCAS II advisory and resolution displays. 


to the TCAS command portion. (See fig. 5.) The 
conventional displays incorporated TCAS symbology 
on the ND that depicted airplane positions, rela- 
tive altitude tags, and vertical direction (if climbing 
or descending). The symbology is defined for pur- 
poses of this experiment in table I. In actual field 
service, the TCAS advisory algorithms have changed 
and their implementation has become more sophis- 
ticated since the inception of this experiment. For 
the conventional displays, the TCAS command to ei- 
ther climb or descend was implemented on the verti- 
cal speed indicator (VSI) as a color-coded command 
bar. (See fig. 5.) The pilot responded by keeping the 
VSI needle in the green portion of the indicator (and 
out of the red). 1 When this response was achieved, 
the pilot was following the TCAS command at an 
appropriate vertical rate. Warnings and commands 
were strictly visual. Auditory displays (which are 
normally a part of TCAS) were considered but were 


J The actual display is in color; those colors are not shown 
herein. 


not employed because they would negate the abil- 
ity to measure spatial awareness differences between 
display formats. 

For the pictorial displays, TCAS symbology was 
implemented in the same manner on the secondary 
see-through ND. However, one important augmenta- 
tion was made to the pictorial scene: the computer- 
generatec image of traffic in the OTW scene was 
also enclosed in a TCAS symbol with the appro- 
priate color- and shape-coded warnings. No reso- 
lution command (i.e., no vertical speed command) 
was presented with the pictorial formats. Full-state- 
variable depiction of the OTW traffic was utilized 
to avoid undesirable discrete updates, which could 
cause undue notice or awareness of traffic. (TCAS 
transponders do not presently encode sufficient state 
information for full-state- variable depiction.) For all 
four display conditions, TCAS was turned off below 
500 ft above ground level (AGL), although unfilled 
blue squares were used to represent other traffic on 
the ND displays (an unfilled black square was used 
on the OTW portion of the pictorial displays), and 
their positions were continually updated. 
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Table I. TCAS II Symbology 


Symbol 0 

Definition 

O Unfilled blue diamond 

Nonthreatening 

♦ Solid blue diamond 

Proximity traffic: 

Within 1200 ft altitude 
6 n.mi. 

Nonthreatening 

# Solid yellow circle 

Traffic advisory: 

Within 1200 ft altitude 
<45 sec 

■ Solid red box 

Resolution advisory: 
Estimated miss 
distance <750 ft 
<30 sec 


a The actual display has these symbols in color; for purposes of 
this report, the symbols are in black and white. 


The pictorial formats discussed above incorporate 
none of the sensor elements associated with synthetic 
vision systems for reduced visibility operations with 
subsonic airplanes or for the lack of forward visi- 
bility with a future high-speed civil transport (in 
which synthetic vision is used instead of lowering the 
nose). However, the other pictorial elements of syn- 
thetic vision are included (pathway representation, 
geographic/feature databases, and dependence on 
advanced navigational aids). Therefore, this paper 
focuses on large-screen, integrated, pictorial displays 
as an approach to synthetic vision technology and 
the problem of optimizing crew spatial awareness. 

Situation Awareness Assessment Tools 
and Techniques 

The assessment of situation awareness is prob- 
ably much more difficult than any attempted defi- 
nition. Several techniques have been suggested in 
the literature, each with advantages and drawbacks. 
The most common method is to measure traditional 
pilot/vehicle performance; however, no direct rela- 
tionship has been established between performance 
and awareness. Therefore, performance measures 
should be supplemented by additional techniques. 
(See refs. 17-21.) The following additional tech- 
niques, compiled from Tenney et al. (ref. 18), were 
considered. 

Think-Aloud Protocols 

Subjects are encouraged to verbalize what they 
are thinking and describe what they are doing and 
why. This technique is somewhat intrusive and is 
utilized only if the subject verbalizes anyway. The 


experimenter takes notes and compares the subject’s 
statements with the subject’s actions. 

Anomalous Cues and Detection Time 

Scenarios are set up that introduce slowly devel- 
oping problems that may require some subject in- 
teraction. The experimenter then measures the time 
elapsed before the subject detects the problem as well 
as the time before any corrective action is taken. 

Freezing and Probing 

This method entails a direct approach in which 
the experimenter either interrupts or “freezes” the 
task, then takes some form of measurement. Usually, 
the experimenter asks relevant questions (in effect, 
probing the subject) concerning that task. (See 
refs. 19 and 20.) Often questions are asked about 
future events (based on what has transpired until 
the moment of task freezing), which may provide 
greater insight about the subject’s awareness of the 
situation at that moment. In other words, the better 
the SA, the more accurately the subject will predict 
the immediate future. In addition, after resuming the 
task, the experimenter may take other measurements 
indicative of SA (such as time to restore to some 
predetermined condition). These methods require 
caution because not only has the original task been 
corrupted, but the probe results must rely on the 
subject’s short-term memory. 

Static Image Flash and Quiz 

Subjects are evaluated for recognition of static 
information, scenarios, or conditions when presented 
over a short period. The more accurately the subject 
perceives or recognizes the situation thus presented, 
the better the SA must be with that particular 
information display system. 

Garden Path and Detection Time 

The subject is led to an erroneous conclusion by 
slowly developing parallel events. Then the experi- 
menter measures the time elapsed before the subject 
detects the mistake in interpretation. (The subject 
is presented information in such a way that a failure 
is correctly realized; however, it is attributed to the 
wrong source.) Scenarios for this technique are more 
difficult to formulate. 

Subjective Methods 

The subject completes questionnaires either ver- 
bally or by handwritten means and expresses per- 
sonal opinions or feelings about the topic. 
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Spatial Awareness Techniques 

For this experiment, several techniques from the 
literature were chosen based upon the ability to gen- 
erate suitable transport approach and landing oper- 
ations. Some of these techniques were successfully 
applied; others were either incorrectly implemented 
or were unsuccessful in providing meaningful results, 
usually because replicates were lacking or because 
statistical control of experimental conditions was in- 
sufficient. Reference 21 addresses the successful and 
unsuccessful applications of these techniques in the 
subject study from the standpoint of effective SA as- 
sessment methodologies. As the focus of this paper 
is on the comparison of integrated pictorial displays 
and conventional EFIS displays, only the success- 
ful techniques (those that yielded meaningful results) 
are discussed herein. 

The traditional lateral and vertical root mean 
square errors were recorded directly during the ba- 
sic or standard task, which was to follow a stan- 
dard approach to landing (SAL). Three SA scenarios, 
which induced new tasks, were implemented within 
the SAL. Two conflicting traffic scenarios were gen- 
erated, one which consisted of crossing traffic sit- 
uations that caused TCAS alerts (the Traffic Con- 
flict Scenario), and the other which involved runway 
blunders by traffic on landing approach to a parallel 
runway (the Runway Blunder Scenario). The third 
scenario, the Path Offset Scenario, exposed each pi- 
lot to incidents of total display system failure fol- 
lowed by simulated display recovery. The pilot’s task 
was, upon display system recovery, to determine the 
own ship location relative to the desired flight path, 
then to return to the flight path in a timely manner. 
Finally, numerous subjective questionnaires were ad- 
ministered in which the subject evaluated the dis- 
plays by answering relevant questions and by rank- 
ing the displays based upon the perception of the 
awareness afforded. Unsolicited subject comments 
were also recorded throughout the trials. Further 
explanations of the individual scenarios, SA eval- 
uation techniques, and measures are in the next 
section. 

Experimental Tasks, Schedule, and 
Questionnaires 

Sixteen pilots were the subjects of the experi- 
ment. All have extensive cockpit experience and 
most are with national commercial airlines. (Three 
are test pilots with commercial airplane manufactur- 
ers.) Four separate experimental tasks were embed- 
ded within the spatial awareness assessment efforts. 
These tasks were induced by scenarios generated to 


exercise the selected SA assessment methods previ- 
ously discussed. These scenarios included the Stan- 
dard Approach to Landing, the Traffic Conflict Sce- 
nario, the Runway Blunder Scenario, and the Offset 
Scenario, all of which were implemented within the 
SAL. 

Standard Approach to Landing 

The Standard Approach to Landing task was 
about 27 n.mi. long and involved a simulated com- 
plex, microwave landing system (MLS) approach 
(fig. 6) to closely spaced, parallel runways. The short 
final approach segment was only 1.7 n.mi. long. The 
SAL, the neighboring traffic routes (fig. 7), and the 
runway configuration (fig. 8) were constructed to pro- 
vide a very complex environment of sufficient dura- 
tion (about 10 min per flight) for exercising the se- 
lected SA measurement tools. The environment was 
not intended to replicate the real world but merely to 
represent a somewhat realistic, demanding future en- 
vironment . Active traffic was included on all routes; 
that is, several airplanes preceded and followed the 
own ship on the basic SAL, a constant stream of traf- 
fic was on the SAL leading to the parallel runway, and 
occasional traffic was on the crossing route. 


BOX v,R APPROACH 


Figure 6. S mulated MLS standard approach to right runway. 

The pilot’s task was to fly the SAL manually (in- 
cluding throttle inputs) using the head-down display. 
Although recognized that conventional EFIS displays 
are not used to fly below decision height altitudes in 
real situations (e.g., 200 ft) without an OTW transi- 
tion, for this investigation the flight ended at the run- 
way threshold without an OTW transition. However, 
all awareness scenarios in the investigation were com- 
pleted well before a 200- ft altitude was reached. The 
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Figure 7. Traffic routes of SAL, parallel approach, and cross- 
ing traffic. 



Figure 8. Offset., parallel runways. 


SAL wa s divided into segments for analysis (fig. 9), 
and the performance metrics for the standard task 
were the traditional lateral and vertical path track- 
ing performances. These metrics are not really spa- 
tial awareness measurements, but they are of related 
interest as they do provide the assurance that any en- 
hanced spatial awareness, as measured by the other 
measurement tools, would not be gained at the ex- 
pense of degraded tracking performance. 

Traffic Conflict Scenario 

The basic approach pattern (to the right par- 
allel runway) always included other aircraft on an 
approach to the left runway. (See figs. 7 and 8.) 
For the Traffic Conflict Scenario, which each pilot 



Figure 9. Segmentation of SAL route for statistical analysis. 

encountered in the data collection session only once 
for each display condition, any one of two aircraft 
on an opposing heading from the own ship on seg- 
ment 2 of the basic SAL (fig. 9) would inexplicably 
(to the subject pilot, but not to the experimenter) 
initiate an altitude maneuver intended to lead to a 
TCAS advisory situation for the own ship. The per- 
formance metrics for this scenario were the detection 
time (from the beginning of the approaching traffic 
altitude maneuver to the pilot’s announced detection 
of the resulting threatening situation) and maneuver 
time (from the beginning of the approaching traf- 
fic altitude maneuver to the initiation of an avoid- 
ance maneuver, if initiated, by the own ship pilot). 
These metrics reflect the supposition that better spa- 
tial awareness would allow earlier detection time, al- 
though earlier maneuver time expectations may not 
be as implicit. Better awareness of the location and 
movement of the other traffic may delay or even elim- 
inate the need for an avoidance maneuver. 

The data run ended after the Traffic Conflict Sce- 
nario without continuing to the threshold. Naturally, 
the effect of prior exposure to this type of scenario 
can be significant; therefore, the pilots became well 
trained for the scenario under all four display condi- 
tions. However, the occurrence of this scenario dur- 
ing the data collection session was infrequent and 
unpredictable. 

Runway Blunder Scenario 

The basic approach pattern for the Runway Blun- 
der Scenario always included another airplane land- 
ing on the left runway 30 sec ahead of the own ship 
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(landing on the right parallel runway). For this sce- 
nario, which each pilot encountered in the data col- 
lection session only once for each display condition, 
the lead aircraft would inexplicably leave the desig- 
nated landing pattern and cross in front of the own 
ship flight path during final approach. (This devia- 
tion would occur while the own ship’s planned alti- 
tude was 400 ft AGL. The TCAS advisory and reso- 
lution logic was turned off below 500 ft, although the 
appropriate displays still presented the traffic with 
unfilled blue squares.) The performance metrics for 
this scenario were the detection time (from the be- 
ginning of the crossing maneuver by the neighboring 
traffic to the pilot’s announced detection of the re- 
sulting threatening situation) and the maneuver time 
(from the beginning of the crossing maneuver to the 
initiation of an avoidance maneuver, if initiated, by 
the own ship pilot). As with the Traffic Conflict Sce- 
nario, better spatial awareness was assumed to al- 
low earlier detection time, although earlier maneu- 
ver time expectations may not be as implicit. Better 
awareness of the location and movement of the other 
traffic may delay or even eliminate the need for an 
avoidance maneuver. 

The effect of prior exposure to this type of sce- 
nario can also be significant; therefore, the pilots 
became well trained for the scenario under all four 
display conditions. As with the Traffic Conflict Sce- 
nario, the occurrence of the Runway Blunder Sce- 
nario during the data collection session was infre- 
quent and unpredictable. Collection of the root mean 
square (rms) tracking data ended before initiation of 
the Runway Blunder Scenario. 

Offset Scenario 

The Offset Scenario exposed each of the 16 pi- 
lots to 4 incidents of simulated recovery from display 
system failure for each display condition. In this sce- 
nario, the standard task was interrupted when the 
display screen was blanked for a significant period, 
after which the original display condition would reap- 
pear (simulating recovery from a main display system 
failure). Upon reappearance, the position of the own 
ship relative to the desired flight path had changed 
(the aircraft had been offset to one of two pre- 
determined positions relative to the planned flight 
path), which thus introduced a new task. The pi- 
lot’s new task in this scenario was to determine the 
location of the own ship relative to the desired flight 
path, then to return to the flight path in a timely 
manner; the pilot was to respond as though the simu- 
lated vehicle were a passenger airliner. Two scenario 
conditions were used: placing own ship 1000 ft above 
and 750 ft to the left of the flight path in segment 4 


of figure 9 (directly in line with approaching traffic 
and in a TCAS resolution situation); the other in- 
volved placing the own ship above and to the right 
of the flight path (again in segment 4, with no threat- 
ening traffic). Two replicates of each scenario condi- 
tion (and thus four offset runs per display condition) 
were used to increase the statistical power. Because 
the airplane heading was not changed for either off- 
set position, the pathway was always in view with 
the pictorial displays upon display system recovery. 

The performance measure for this scenario was 
recovery time. A return to flight path was defined 
as achievement of an error of less than half a dot in 
lateral and vertical tracking and a heading error of 
less than 5°. Better spatial awareness was assumed 
to allow earlier position determination and result in a 
shorter recovery time. For this scenario, the standard 
task was interrupted in segment 3 of figure 9, and 
the offset placed the airplane in segment 4. The 
standard rms tracking performance measures were 
not gathered for segments 3 and 4 during an approach 
that included the Offset Scenario. However, tracking 
data collection was resumed after path recovery for 
the remaining segments of the flight (segments 5 7). 

Schedule 

Table II presents a typical 2-day schedule for a 
pilot participating in the experiment. After being 
briefed on the purpose of the experiment, the details 
of each display condition, and the various scenario 
conditions, the pilot was allowed about 20 min to be- 
come familiarized with the handling characteristics of 
the airplane model in unstructured flight maneuvers. 
Half the pilots used the conventional EFIS without 
the flight director display condition for this purpose; 
the other half used the 70° pictorial display condi- 
tion. The pilots were thoroughly trained with the 
standard approach task, then were thoroughly ex- 
posed to the scenario condition for each display con- 
dition. The second day was the data collection ses- 
sion. The display conditions were randomly blocked 

Table II. Spatial Awareness Schedule 

Day 1 («10 hr) 

Briefing session 
Training session: 

Characteristics familiarization 
Display conditions 1 -4 

Day 2 (wlO hr with rest periods) 

Data collection: 

Display conditions 1-4 
Questionnaires 
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Table III. Data Collection Session 


Display 

condition 

Approach 

conditions 0 

Questionnaires | 

Intrusion 

Display 

evaluation 

Display 

comparison 

3 

R\ 0 4 , Oi, R, 0 3 , T, R, 0 2 

X 

X 


1 

R, Oi, T, R, 0;j, 0 4 , R*, 0 2 

X 

X 


2 

R, 0 3 , T, R, 0 2 , R*, 0 4 , Oi 

X 

X 


4 

R, 0 2 , R*, 0 4 , Oi, R, 0 3 , T 

X 

X 

X 


“Conditions: 

R signifies a standard approach. 

R* signifies a standard approach with runway blunder. 
T signifies a traffic avoidance maneuver. 

O n signifies an offset occurrence. 


across pilots, and the experimental tasks were ran- 
domized within each display condition. Table III 
presents an outline of a typical session, the details 
of which varied from pilot to pilot. 

Questionnaires 

As shown in table III, each pilot was asked to 
complete two questionnaires at the end of the data- 
gathering runs for each display condition. The first 
questionnaire probed specific items concerning the 
traffic scenarios encountered with that display condi- 
tion, and the second dealt with the evaluation of that 
display concept in general. After completing all runs 
and the individual display concept questionnaires for 
each display condition, the pilots completed a final 
questionnaire that involved detailed comparisons of 
the four display concepts. 

Experimental Results and Discussion 

The four scenarios were designed as full- factorial, 
within-subjects experiments, with pilots, display con- 
dition, any scenario conditions, and any replicates as 
the factors. Extensive pilot variability is expected; 
therefore, pilot variability was isolated from the rest 
of the analyses by its inclusion as a main factor in the 
experiments. The data collected in the experiments 
were analyzed using univariate analyses of variance 
for each metric. Newman-Keuls tests (discussed in 
ref. 22) of individual means were performed at vari- 
ous stages in the analyses. (All such tests were made 
at a 1-percent significance level.) The objective re- 
sults are presented and discussed for each scenario, 
and some subjective results are discussed thereafter. 

Traffic Conflict Scenario 

The Traffic Conflict Scenario exposed each pilot 
to one of two similar traffic avoidance situations for 


each display condition. No replication was used; 
therefore, the only factors in the experimental design 
were the pilots and the display conditions. 

Table IV summarizes the analysis results for de- 
tection time and maneuver time. Figures 10 and 11 
graphically present the results of the Traffic Con- 
flict Scenario. All 16 pilots detected each threat- 
ening situation, regardless of the display condition. 
(See fig. 10.) However, the differences between the 
detection times for the EFIS display conditions 
and the pictorial display conditions (fig. 11, about 
10 sec) were statistically significant. Differences 
within the display types (EFIS and pictorial) were 
not significant . 


Table IV. Analysis of Variance Results for Traffic 
Conflict Scenario 

[From figures 10 and 11] 


Factor 

Degrees 
of freedom 

Significance 0 

Detection time 

Pilots 

15 

* 

Display 

3 

** 

Error 

45 

- 

Total 

63 

- 

Maneuver time 

Pilots 

14 

- 

Display 

3 

- 

Error 

20 

- 

Total 

6 37 

- 


“Significance: 

-Not significant at levels considered. 
* Significant at 5- percent level. 
♦♦Significant at 1-percent level. 
^Missing 26 cases. 
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□ Maneuvered 
■ Detected 



Display 

Figure 10. Traffic Conflict Scenario number of detections and 
subsequent maneuvers. 



figure 1 1. Traffic Conflict Scenario mean detection times per 
display concept and condition. 


The alt itude maneuver executed by the approach- 
ing traffic usually resulted in a TCAS advisory or a 
TCAS resolution, with the outcome dependent upon 
the current tracking performance of the own ship. 
Thirty-eight own ship avoidance maneuvers were ex- 
ecuted (fig. 10); in 26 cases the pilot decided not to 
execute a maneuver. The maneuvers may have re- 
sulted from a TCAS resolution or from an indepen- 
dent decision of the own ship pilot. The no-maneuver 
decisions may have been made because the situation 
was judged not serious. The 23 no-maneuver cases 
with the pictorial displays can be attributed to such 
judgments, as no TCAS resolution was presented to 
the pilots and the requirement for a maneuver was 
wholly their decision. Detailed analysis of the three 
no-maneuver cases with the conventional displays re- 
vealed that TCAS resolutions were presented. How- 
ever, in each case the pilot’s response to the TCAS 
command came so late that the maneuver detection 
logic used for scoring was not tripped before the run 


ended. In any case, the analysis of variance for the 
maneuvei time measure found no statistically signifi- 
cant diffe ences for any factors of the scenario exper- 
iment (and no figures are presented). 

The inference from these results is that the pic- 
torial displays provided the pilot with better traffic 
information than did the EFIS displays. Detection 
of the threatening traffic situations occurred earlier 
and at greater distances (the 10-sec-earlier detection 
time translates into 1 n.mi. of increased separation) 
with the pictorial displays; and with the increased 
awareness of the situation, the pilots initiated fewer 
avoidance maneuvers. 

Runway Blunder Scenario 

The Runway Blunder Scenario exposed each pilot 
to one incident for each display condition in which 
another airplane landing on the left parallel runway 
30 sec ahead of the own ship inexplicably (to the 
subject pilot, but not to the experimenter) crosses 
the own ship flight path on final approach to the right 
runway. Figure 12 illustrates the Runway Blunder 
Scenario and the obvious visual advantages of the 
pictorial display formats. No replication was used; 
therefore, only pilot factors and display conditions 
were analyzed. 

Table V summarizes the results of the analyses 
for detect on time and maneuver time. With the 
EFIS displays, about half the blunders were de- 
tected. ( i he map scale was always on maximum 
on final approach.) With the pictorial displays, the 


lable V. Analysis of Variance Results for Runway 
Blunder Scenario 


Factor 

Degrees 
of freedom 

Significance" 

Detection time 

Pilots 

15 

_ 

Display 

3 

** 

Error 

30 

_ 

Total 

b 48 

. 

Maneuver time 

Pilots 

12 

* 

Display 

3 

** 

Error 

15 

_ 

Tota 

e 30 

- 


"Significance: 

-Not significant at levels considered. 
^Significant at 5-percent level. 
**Signifirant at 1-percent level. 
^Missing 15 cases. 

^Missing c3 cases. 
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(a) Conventional EFIS with flight director display format (b) Conventional EFIS with flight director display format 
with incurring traffic (unfilled blue diamond in color with incurring traffic (unfilled blue diamond in color 

version) near beginning of incursion maneuver. version) near end of incursion maneuver. 



L-93-661 L-93-660 


(c) Portion of pictorial display format with incurring traffic (d) Portion of pictorial display format with incurring traffic 

(airplane silhouette enclosed by black square) near begin- (airplane silhouette enclosed by black square) near end of 

ning of incursion maneuver. incursion maneuver. 

Figure 12. Runway Blunder Scenario depicting parallel traffic incursions. 


16 pilots detected all threatening situations. (See 
fig. 13.) The differences between the mean detection 
times for the EFIS display conditions and the picto- 
rial display conditions (fig. 14, about 8 sec) were sta- 
tistically significant. The difference within the EFIS 
display types (3.3 sec sooner for the flight director 
condition versus without the flight director) also was 
significant, whereas the difference between the picto- 
rial conditions (0.7 sec sooner for the 40° condition) 
was not. 


Of the 64 runway blunder incidents, 15 went un- 
detected under the EFIS display conditions. (See 
fig. 13.) Within the 49 detected incidents, the pilots 
chose to initiate a go-round maneuver in 31 cases. 
Analysis of the maneuver time measure for those 
31 cases revealed significant differences between most 
paired means comparisons. (See fig. 15.) The ma- 
neuver time difference between the EFIS displays 
was statistically significant, with the flight director 
mean 4.6 sec earlier than the EFIS without the flight 
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Conventional 


Flight director 40° pictorial 
Display 


70° pictorial 


Figure 13. Runway Blunder Scenario number of detections 
and subsequent maneuvers. 
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Conventional Flight director 40° pictorial 
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Figure 14. Runway Blunder Scenario mean detection times 
per display concept and condition. 



Figure 15. Runway Blunder Scenario mean times to maneuver 
since detection. 


director mean. The pictorial display mean of 40° was 
a significant 3.4 sec earlier than the flight director 
mean. The difference between the 40° and the 70° 
pictorial display means (2.0 sec earlier with the 70° 
pictorial display) was not statistically significant. 


The inference from these results is that the pic- 
torial displays provided the pilot with better traffic 
awareness near the runway than did the EFIS dis- 
plays. Fifteen of the 32 runway blunder incidences 
went undetected with the EFIS displays; detection in 
the remaining 17 incidents came later than with the 
pictorial displays. Also, with the increased awareness 
of the runway situation, the pilots initiated fewer go- 
round maneuvers when they used the pictorial dis- 
plays. Within the EFIS display conditions, the detec- 
tion time and the maneuver time means were lower 
for the EFIS flight director condition. Without the 
flight director, the pilot is probably intent on inter- 
preting the raw error information and controlling to 
minimize glide slope and localizer deviations during 
final approach. Less time would thus be available to 
monitor the neighboring traffic than when just fol- 
lowing the flight director commands. 

Offset Scenario 

The Offset Scenario exposed each pilot to four 
incidents of simulated recovery from display system 
failure for each display condition. The pilot’s task 
in this scenario was, upon display system recovery, 
to determine the location of the own ship relative to 
the desired flight path, then to return to the flight 
path in a timely manner. Two scenario conditions 
were used, one placing the own ship in a TCAS 
resolution situation and the other in an unthreatened 
position. Two replicates of each scenario condition 
were used; and therefore, the factors analyzed were 
pilots, displays, scenario condition, and replicates. 

Table VI summarizes the results of the analyses 
for recovery time. (A return to path was defined 
as achievement of an error of less than half a dot 
in lateral and vertical tracking and a heading error 
of less than 5°.) Statistically significant differences 
were founc between the displays and the interaction 
between the displays and the scenario conditions. 
Figure 16 presents the results for the display factor 
and figure 17 graphically presents the results for the 
second-order interaction between the displays and 
the scenario conditions. In figure 16, more time is re- 
quired to recover when using the conventional EFIS 
displays w thout the flight director. With the flight 
director, the recovery time was 14.6 sec quicker than 
without the flight director, and the performances 
with the pi ctorial displays were at least 10.2 sec faster 
than the flight director results; these differences were 
statistically significant. The difference between the 
pictorial conditions (2.5 sec faster for the 40° condi- 
tion) was not significant. 

Figure 17 shows that with the two conventional 
EFIS display conditions, the pilots took longer to 
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Table VI. Analysis of Variance Results for Offset Scenario 


Factor 

Degrees of 
freedom 

Significance 0 of 
recovery time 

Pilots 

15 

** 

Displays 

3 

** 

Conditions 

1 

- 

Replicates 

1 

- 

Pilots x Displays 

45 

- 

Pilots x Conditions 

15 

- 

Displays x Conditions 

3 

* 

Pilots x Displays 
x Condition 

45 


Error 

127 

- 

Total 

255 



a Significance: 

-Not significant at levels considered. 
*Significant at 5-percent level. 
^^Significant at 1-percent level. 
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Display 


Figure 16. Offset Scenario mean times to recover to intended 
flight path per display concept and condition. 



Figure 17. Offset Scenario second-order interaction between 
displays and scenario conditions; mean times to recover 
to intended flight path per display concept and condition. 


recover from the left offset condition (the TCAS traf- 
fic resolution case) than from the right offset condi- 
tion. For the conventional EFIS without the flight di- 
rector, the difference was a significant 13 sec, whereas 
the significant difference for the flight director case 
was about 6 sec. The differences for the two pictorial 
cases were not statistically significant. 

The inference from these results is that the pilots 
were able to determine the own ship location rela- 
tive to the desired flight path and return to the flight 
path more quickly with the pictorial displays. The 
flight director recovery was faster than the conven- 
tional display without the flight director, probably 
because interpreting the raw error information was 
more time-consuming than just following the flight 
director commands. The difference in recovery time 
between the pictorial displays and the flight director 
display was attributed to better spatial awareness, 
but it might also involve more aggressive manual in- 
tercepts of the flight path with the pictorial displays 
versus the intercept logic within the flight director. 
Perhaps a better spatial awareness metric for this 
scenario would have been maneuver time (the time 
elapsed before maneuvering began, as was used in 
the Runway Blunder scenario). 

The statistically meaningful results from the sce- 
nario conditions within the Offset Scenario (the 
second-order interaction term) occurred with the 
conventional EFIS displays. When the display sys- 
tem recovered from the simulated display failure 
to reveal a TCAS resolution situation (the left off- 
set condition), the pilots probably responded to the 
TCAS vertical resolution before attempting to de- 
termine the own ship location relative to the desired 
flight path and returning to the flight path. There- 
fore, recovery took longer for the left offset condition 
than for the right. With the pictorial displays, the 
sense of urgency to move is much higher for the left 
offset condition, and the direction of desired move- 
ment is readily determined from the visual presenta- 
tion. (The maneuvering response to a TCAS resolu- 
tion under the pictorial display conditions was left to 
the pilot’s discretion, as opposed to the EFIS TCAS 
resolution of vertical movement.) Therefore, recovery 
was initiated more quickly for the left offset condition 
than for the right (offset condition with no threaten- 
ing traffic), although the 3- to 3.5-sec differences were 
not statistically significant. 

Subjective Results 

Table III enumerates the nine questionnaires 
administered to each pilot. The summary of those 
subjective results (which is sufficient for this paper) 
revealed a dramatic improvement in all aspects of 
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Very Somewhat Neutral Somewhat Very 
hard hard easy easy 


(as a sanity check, the same question), the ease 
of maintaining spatial awareness when using each 
display configuration (without comparison to the 
other display configurations). In both instances, the 
two pictorial formats dramatically improved spatial 
awareness and, in particular, when the large-screen 
70° version was involved. 

Another subjective assessment of each display 
configuration (without comparison to the other dis- 
play configurations) used the Modified Cooper- 
Harper scale (modified to extend its utility beyond 
handling quality evaluations; ref. 23) that is showm in 
figure 19. Figure 20 presents the average, maximum, 
and minimum ratings (not plus or minus the stan- 
dard deviations) for all experiment scenarios. The 
two pictorial formats distinctly improved the mod- 
ified rating (in both mean rating and spread), al- 
though differences between the two fields of view were 
hard to assess within the confinements of lower end 
of the scale. 


(a) Ease of becoming disoriented. 



Very Somewhat Neutral Somewhat Very 

hard hard easy easy 


(b) Ease of maintaining spatial awareness. 

Figure 18. Rating results from pilots for two subjective 
categories. 

spatial awareness when both pictorial formats were 
used and, in particular, when the large-screen 70° 
version was involved. Figure 18 presents rating re- 
sults for two subjective categories as typical exam- 
ples. The pilots were asked to rate, the ease of be- 
coming disoriented and, in an opposite connotation 


Figure 21 presents the results of comparative rank 
ordering by the pilots for several categories on a scale 
of 1 (the most desirable display) to 10 (the least desir- 
able display). The average, maximum, and minimum 
rankings presented (not plus or minus the standard 
deviations). The categories are used to compare the 
display concepts over all the scenarios. The pilots 
ranked display effectiveness for success in monitoring 
traffic, foi reduction of their workload, and for their 
reactions regarding the entire experiment. Again, 
based on the subjective comments, the pictorial for- 
mats substantially improved all aspects of spatial 
awareness (in both average ranking and spread). In 
particular the large-screen 70° version was preferred 
by 14 of the 16 pilots; two pilots had no preference. 

In adc ition to the formal questionnaire results, 
other subjective comments were obtained. Particu- 
larly notable are the following: 

“Like flying on a beautiful VFR [visual flight 
rules] day ” 

“Provides immediate assessment of the 
situation. . . 

“Ability to fly complex approaches is greatly 
improved. ’ 

“Easier to detect traffic incursions and runway 
blunders. 1 

“Display of pictorial world is natural and easy to 
interpret. 1 

Objective Tracking Performance Results 

In addition to the SA measurement techniques, 
standard rms tracking data were collected and 
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Figure 19. Modified Cooper-Harper scale for mental workload assessment. 



Figure 20. Modified Cooper- Harper mean rating results over all experiment scenarios. 
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Figure 21. Comparative rank ordering by pilots or three categories. 


analyzed. The standard or basic SAL was divided 
into segments for this purpose (fig. 9), with the fol- 
lowing designations: 


Segment 

Description 

1 

Turning entry 

2 

Straight descent 

3 

Descending turn 

4 

Transition to straight and level, deceleration 

5 

Level turn 

6 

3° descent, deceleration, and turn final 

7 

Short final approach 


The analysis for segment 1 (entry to the SAL from 
the off-path initial conditions) was not meaningful in 
terms of spatial awareness or display format results 
and is not presented. Table VII presents a summary 
of the analyses of variance results for the rms of the 


vertical and lateral tracking errors for the other seg- 
ments. Statistically significant differences were found 
between pilots, displays, and the interaction of those 
two factors. Pilot response varied greatly (as ex- 
pected); t lerefore, pilot variability was isolated from 
the rest of the analyses by its inclusion as a main fac- 
tor in the analyses. The pilot-by-display interaction 
typically indicated that the pilots reacted differently 
to the display effect; that is, one pilot might exhibit 
a very large difference during two display conditions, 
whereas another pilot would exhibit a smaller dif- 
ference. The differences were typically in the same 
direction, with only the magnitudes varying among 
pilots. 

Figure 22 presents a comparison of the display 
condition rms lateral error means from the 16 pilots 
for each segment. Not surprisingly, the lateral track- 
ing performance error was significantly larger for ev- 
ery segment of the SAL in which the conventional 
EFIS was used without the flight director display. 
Differences between the conventional EFIS with the 
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Tabic VII. Significance of Variance Results for Tracking 
Performance Measures 


(a) Summary of analyses for segments 2, 5, and 6 


Factor 

Degrees 
of freedom 

Significance" of rms 
performance for 

Vertical 

Lateral 

Pilots 

15 

** 

** 

Displays 

3 

** 

** 

Replicates 

6 

- 

- 

Pilots x Displays 

45 

** 

** 

Error 

378 



Total 

447 




(b) Summary of analyses for segments 3 and 4 


Factor 

Degrees 
of freedom 

Significance" of rms 
performance for 

Vertical 

Lateral 

Segment 3 

Pilots 

15 

** 

** 

Displays 

3 

** 

** 

Replicates 

2 

* 

- 

Pilots x Displays 

45 

** 

** 

Error 

126 



Total 

191 



Segment 4 

Pilots 

15 

** 

** 

Displays 

3 

** 

** 

Replicates 

2 

- 

- 

Pilots x Displays 

45 

** 

** 

Error 

126 



Total 

191 




(c) Summary of analysis for segment 7 


Factor 

Degrees 
of freedom 

Significance" of rms 
performance for — 

Vertical 

Lateral 

Pilots 

15 

** 

** 

Displays 

3 

** 

** 

Replicates 

5 

- 

- 

Pilots x Displays 

45 

** 

** 

Error 

315 



Total 

383 




Significance: 

-Not significant at levels considered. 
*Significant at 5-percent level. 
**Significant at 1- percent level. 


flight director and the two pictorial display condi- 
tions were also significant (flight director error was 
larger), whereas differences between the 40° and 70° 
display conditions were not. Differences in perfor- 
mance between segments for a particular display con- 
dition can be attributed to the type of segment . (Seg- 
ments 3, 5, and 6 included turns and segments 2, 4, 
and 7 were straightaways.) 

Figure 23 presents a comparison of the display 
condition rins vertical error means from the 16 pilots 
for each segment. Not surprisingly, the vertical 
tracking performance error was significantly larger 
for every segment for the conventional EF1S without 
the flight director display. Differences among the 
conventional EFIS with the flight director and the 
two pictorial display conditions were also significant, 
but only for segments 2, 3, and 6 (flight director 
error was larger). Differences between the 40° and 
the 70° display conditions were not significant for 
any segment. Differences in performance between 
segments for a particular display condition can be 
attributed to the type of segment (segments 2, 3, 6, 
and 7 included descents, whereas segments 4 and 5 
were level). 

The lateral and vertical tracking performances 
with the pictorial display conditions were at least 
as good as or better than the performances with 
the EFIS flight director display, and the flight direc- 
tor performances were much better than the EFIS 
without the flight director. Although no conclusions 
may be drawn from these facts about increased or 
decreased spatial awareness, they do provide the as- 
surance that the increased spatial awareness from the 
pictorial displays, as measured by the other measure- 
ment tools, did not degrade tracking performance. 

Inferences From Results 

Conclusive inferences can be drawn from the ob- 
jective and subjective results. Meaningful compar- 
isons are possible between the two EFIS display for- 
mats, between the two pictorial display formats, and 
between the conventional EFIS displays and the pic- 
torial displays. 

EFIS Comparisons 

In all cases in which objective or subjective com- 
parisons were possible, EFIS displays with the flight 
director achieved either equivalent or better per- 
formance. The earlier supposition had been that 
concentration only on centering the flight direc- 
tor needles might reduce the pilot’s awareness of 
surrounding events, whereas using raw position er- 
rors might increase spatial awareness. However, that 
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rms of vertical error rms of lateral error 


Straight Descending Transition to Level turn 3° descent, Short final 

descent turn straight and level, decel, and approach 

Hor'ol ti irn 



Segment 2 Segment 3 Segment 4 Segment 5 Segment 6 Segment 7 


Conventional 
Flight director 
40° pictorial 
70° pictorial 

Figure 22. Mean lateral errors (for all pilots) per display concept md condition for each path segment. 
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11 40° pictorial 
| 70° pictorial 


Figure 23. Mean vertical errors (for all 


pilots) per display concept and condition for each path segment. 
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supposition was proved invalid here. Perhaps the key 
word in that supposition was the word only. Better 
spatial awareness appears to be gained because the 
flight director imposes a lower path-tracking work- 
load on the pilot, thereby allowing time to scan other 
sources of information besides the flight director nee- 
dles. Use of raw data error in the EFIS without the 
flight director requires that almost constant attention 
be devoted to the path-tracking task. 

Pictorial Comparisons 

The objective data revealed equivalent perfor- 
mance for the 70° pictorial display compared with the 
40° FOV. The subjective data, however, revealed a 
strong preference for the wider FOV, particularly for 
awareness during turning entry and traffic situations. 

EFIS and Pictorial Comparisons 

Both the objective and subjective data demon- 
strated that the integrated pictorial displays in- 
creased spatial awareness over the conventional EFIS 
display formats. 

Concluding Remarks 

An extensive simulation study was performed 
to determine and compare the spatial awareness of 
commercial airline pilots on simulated landing ap- 
proaches using conventional flight displays with their 
awareness using advanced pictorial ‘'pathway in the 
sky 7 ' displays. Various situation awareness measure- 
ment techniques were incorporated within the sce- 
narios. Conflicting traffic situation assessments and 
flight path recaptures after recovery from simulated 
display system failure were created to assess spa- 
tial awareness using different display formats. Both 
objective and subjective techniques were employed. 
The spatial awareness scenarios yielded results that 
were consistent across and within the objective and 
subjective measures. 

The objective data analyses revealed that bet- 
ter spatial awareness performance was achieved when 
the flight director was included in the electronic flight 
information systems (EFIS) display. However, the in- 
tegrated pictorial displays consistently provided sub- 
stantially increased spatial aw T areness compared with 
either of the conventional EFIS display formats. Ob- 
jective data results for the wider field-of-view (FOV) 
pictorial display w'ere equivalent to those obtained 
with the narrower pictorial format. 

The subjective results, which were also summa- 
rized herein, indicated a strong preference for the 
flight director within the EFIS displays. However, 


the subjective study revealed that all aspects of spa- 
tial awareness were dramatically improved by the two 
pictorial formats and, in particular, by the large- 
screen version with a 70° FOV. 

Therefore, integrated pictorial displays show sig- 
nificant promise for improving situation awareness. 
These types of formats also can be the basis for 
an effective synthetic vision system, one which 
can solve restricted visibility problems associated 
with advanced subsonic and future high-speed civil 
transports. 

NASA Langley Research Center 
Hampton, VA 23681-0001 
July 1, 1991 
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